{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "0ctRYDdUyshU"
   },
   "source": [
    "# [xTuring](https://github.com/stochasticai/xturing) - LLaMA INT8 efficient fine-tuning tutorial\n",
    "\n",
    "This tutorial aims to show how easy it is to perform fine-tuning with xTuring. This notebook shows how to fine-tune LLaMA 7B model on GPU which has limited memory, it requires only 9GB VRAM"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "DfZriB2ez6kp"
   },
   "source": [
    "## 1. Install the `xTuring` library"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "bNFVFKld2Ju4"
   },
   "outputs": [],
   "source": [
    "!pip install xturing --upgrade"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0TwnHA9zz9v6"
   },
   "source": [
    "## 2. Download and unzip the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "KRO0TVPP2iFy"
   },
   "outputs": [],
   "source": [
    "!wget https://d33tr4pxdm6e2j.cloudfront.net/public_content/tutorials/datasets/alpaca_data.zip\n",
    "!unzip alpaca_data.zip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tCFfBj2n0CBw"
   },
   "source": [
    "## 3. Load the dataset and initialize the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "pwvVo7Sx2vVj"
   },
   "outputs": [],
   "source": [
    "from xturing.datasets.instruction_dataset import InstructionDataset\n",
    "from xturing.models import BaseModel\n",
    "\n",
    "instruction_dataset = InstructionDataset(\"/content/alpaca_data\")\n",
    "# Initializes the model\n",
    "model = BaseModel.create(\"llama_lora_int8\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GPIwmGIp1yPz"
   },
   "source": [
    "## 4. Start the finetuning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "background_save": true
    },
    "id": "BHa8P_jM7jCx"
   },
   "outputs": [],
   "source": [
    "# Finetuned the model\n",
    "model.finetune(dataset=instruction_dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pp6iDEYxesRe"
   },
   "source": [
    "## 5. Generate an output text with the fine-tuned model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "lYfFHtW33j4B"
   },
   "outputs": [],
   "source": [
    "# Once the model has been finetuned, you can start doing inferences\n",
    "output = model.generate(texts=[\"Why LLM models are becoming so important?\"])\n",
    "print(\"Generated output by the model: {}\".format(output))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3CJgRvFVevwK"
   },
   "source": [
    "## Do you have any questions?\n",
    "\n",
    "You can open an issue in our [GitHub repo](https://github.com/stochasticai/xturing) \n"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "provenance": []
  },
  "gpuClass": "standard",
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
